Appendix F to GM 21.A.101 The use of service experience in the exception process
CAA ORS9 Decision No. 1
F.1 Introduction.
Service experience may support the application of an earlier certification specification pursuant to point 21.A.101(b)(3) if, in conjunction with the applicable service experience and other compliance measures, the earlier certification specification provides a level of safety comparable to that provided by the latest certification specification. The applicant must provide sufficient substantiation to allow the CAA to make this determination. A statistical approach may be used, subject to the availability and relevance of data, but sound engineering judgment must be used. For service history to be acceptable, the data must be both sufficient and pertinent. The essentials of the process involve:
— A clear understanding of the certification specification change and the purpose for the change,
— A determination based on detailed knowledge of the proposed design feature,
— The availability of pertinent and sufficient service experience data, and
— A comprehensive review of that service experience data.
F.2 Guidelines.
The CRI process (either as a stand-alone CRI or included in the CRI A-01) would be used, and the applicant should provide documentation to support the following:
F.2.1 The identification of the differences between the certification specification in the existing basis and the certification specification as amended, and the effect of the change to the specification.
F.2.2 A description as to what aspect(s) of the latest certification specifications the proposed changed product would not meet.
F.2.3 Evidence showing that the proposed certification basis for the changed product, together with applicable service experience, relative to the hazard, provides a level of safety that approaches the latest certification specification, yet is not fully compliant with the latest certification specifications.
F.2.4 A description of the design feature and its intended function.
F.2.5 Data for the product pertinent to the requirement.
F.2.5.1 Service experience from such data sources, such as:
— Accident reports,
— Incident reports,
— Service bulletins,
— Airworthiness directives,
— Repairs,
— Modifications,
— Flight hours/cycles for fleet leader and total fleet,
— World airline accident summary data,
— Service difficulty reports,
— Accident Investigation Board reports, and
— Warranty, repair, and parts usage data.
F.2.5.2 Show that the data presented represent all relevant service experience for the product, including the results of any operator surveys, and is comprehensive enough to be representative.
F.2.5.3 Show that the service experience is relevant to the hazard.
F.2.5.4 Identification and evaluation of each of the main areas of concern with regard to:
— Recurring and/or common failure modes,
— Cause,
— Probability by qualitative reasoning, and
— Measures already taken and their effects.
F.2.5.5 Relevant data pertaining to aircraft of similar design and construction may be included.
F.2.5.6 Evaluation of failure modes and consequences through analytical processes. The analytical processes should be supported by:
— A review of previous test results,
— Additional detailed testing as required, or
— A review of aircraft functional hazard assessments (FHA) and any applicable system safety assessments (SSA) as required.
F.2.6 A conclusion that draws together the data and the rationale.
F.2.7 These guidelines are not intended to be limiting, either in setting the required minimum elements or in precluding alternative forms of submission. Each case may be different, based on the particulars of the system being examined and the requirement to be addressed.
F.3 Example: 25.1141(f) for Transport Category Aeroplanes.
NOTE: This example is taken from the FAA’s certification experience, so references to FAR sections and amendments are kept.
F.3.1 The following example, for transport category aeroplanes (§ 25.1141(f), APU Fuel Valve Position Indication System), illustrates the typical process an applicant follows. The process will be the same for all product types.
F.3.2 This example comes from a derived model transport aeroplane where significant changes were made to the main airframe components, engines and systems, and APU. The baseline aeroplane has an extensive service history. The example shows how the use of service experience supports a finding that compliance with the latest certification specifications would not contribute materially to the level of safety and that application of the existing certification basis (or earlier amendment) would be appropriate. The example is for significant derived models of transport aeroplanes with extensive service history. It illustrates the process, following the guidelines in this Appendix, but does not include the level of detail normally required.
F.3.2.1 Determine the differences between the certification specifications applied in the original certification basis and the latest certification specification, and the effect of the change to the certification specifications. The original certification basis of the aeroplane that is being changed is the initial release of Part 25. Amendment 25-40 added requirement § 25.1141(f), which mandates that power-assisted valves must have a means to indicate to the flight crew when the valve is in the fully open or closed position, or is moving between these positions. The addressed hazard would be risk of APU fire due to fuel accumulation caused by excessive unsuccessful APU start attempts.
F.3.2.2 What aspect of the proposed changed product would not meet the latest certification specifications? The proposed APU fuel valve position indication system does not provide the flight crew with fuel valve position or transition indication and, therefore, does not comply with the requirements of § 25.1141(f).
F.3.2.3 The applicant provides evidence that the proposed certification basis for the changed product, together with applicable service experience of the existing design, provide a level of safety that approaches, yet is not fully compliant with, the latest certification specifications. The APU fuel shut-off valve and actuator are unchanged from those used on the current family of aeroplanes, and have been found to comply with the earlier Amendment 25-11 of § 25.1141. The existing fleet has achieved approximately (#) flights during which service experience of the existing design has been found to be acceptable. If one assumes a complete APU cycle, i.e. start-up and shutdown for each flight, the number of APU fuel shut-off valve operations would be over 108 cycles, which demonstrates that the valve successfully meets its intended function and complies with the intent of the certification specification.
F.3.2.4 The applicant provides a description of the design feature and its intended function. The fuel shut-off valve, actuator design, and operation is essentially unchanged with the system design ensuring that the valve is monitored for proper cycling from closed to open at start. If the valve is not in the appropriate position (i.e. closed), then the APU start is terminated, an indication is displayed on the flight deck, and any further APU starts are prevented. Design improvements using the capability of the APU electronic control unit (ECU) have been incorporated in this proposed product change. These design changes ensure that the fuel valve indication system will indicate failure of proper valve operation to the flight crew, and these features increase the level of functionality and safety, but the system does not indicate valve position as required by § 25.1141(f).
F.3.2.5 The FAA and the applicant record this in an issue paper. The FAA can use the G-1 or a technical issue paper for this purpose. An issue paper was coordinated, included data, or referenced reports documenting relevant service experience compiled from incident reports, fleet flight hour/cycle data, and maintenance records. The issue paper also discussed existing and proposed design details, failure modes, and analyses showing to what extent the proposed aeroplane complies with the latest amendment of § 25.1141. Information is presented to support the applicant’s argument that compliance with the latest amendment would not materially increase the level of safety. Comparative data pertaining to aircraft of similar design and construction are also presented.
F.3.2.6 The conclusion, drawing together the data and rationale, is documented in the G-1 issue paper. The additional features incorporated in the APU fuel shut-off valve will provide a significant increase in safety to an existing design with satisfactory service experience. The applicant proposes that compliance with the latest amendment would not materially increase the level of safety and that compliance with § 25.1141 at Amendment 25-11 would provide an acceptable level of safety for the proposed product change.